Repository navigation
[NVIDIA][AgentX] MiniMax-M3 B200 vLLM digest pin, TP4 DRAM-offload tuning and capped-active c10 entries / [NVIDIA][AgentX] MiniMax-M3 B200 vLLM 按摘要固定镜像、调优 TP4 DRAM 卸载并新增限制活跃序列的 c10 配置 - #4382
Conversation
…ning and capped-active c10 entries Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: hshrivastava-droid <hshrivastava@nvidia.com>
|
Thanks for the contribution!
中文感谢你的贡献!
|
There was a problem hiding this comment.
Beyond the inline findings, I checked two more things. First, the selectors override_tp4_c15 and override_tp4_c32_dram aren't touched in this diff, but they still exist in agentic.yaml (lines 108 and 204), so they resolve. Second, the perf-changelog.yaml change only appends one block at the end of the file, covering both config keys, and leaves the existing entries alone.
Extended reasoning...
Both checks were run against the diff: the two unchanged override selectors are present in the recipe, and perf-changelog.yaml only adds one 10-line block at the tail.
Findings marked 🟡 are optional suggestions and need no follow-up push.
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [20], srt-recipe: agentic.yaml:override_cap4_tp4_c20_dram } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [20], srt-recipe: agentic.yaml:override_cap3_tp4_c20_dram } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [32], srt-recipe: agentic.yaml:override_cap6_tp4_c32_dram } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [32], srt-recipe: agentic.yaml:override_cap16_tp4_c32_dram } |
There was a problem hiding this comment.
🔴 Two pairs of new points can't be told apart by the tooling: cap4/cap3 at c20 and cap6/cap16 at c32. Their results collide, so reuse validation and per-experiment reruns fail. The c10 key reuses TP4 with the same conc and offload settings, and only the srt-recipe override differs. Exp names (generate.py:1062) and agentic_key (results/artifacts.py:123) both ignore the override. Fix: give every point a distinct identity, for example by keeping one capped variant per (tp, conc, offload), or by adding the recipe or override to the exp name and to agentic_key.
Why this was flagged
In minimaxm3-fp4-b200-vllm-agentic-mtp-c10, nvidia-master.yaml:5550-5551 are both tp 4, conc 20, dram, vllm-simple. Lines 5552-5553 are both tp 4, conc 32, dram, vllm-simple, and so is override_tp4_c32_dram at :5531 in the main key, on the same runner cluster:b200-nscale. generate.py:1062-1066 builds exp_name only from model_code, tp, conc, offload suffix and spec, so each pair gets the same exp_name. filter_exp_names (generate.py:1437-1441) then raises "Experiment name(s) matched multiple rows" when a test-config run targets either point. For single-node runs, agentic_key (results/artifacts.py:166-180) keys on hw, model prefix, framework, precision, tp, pp, dcp, pcp, ep, dp_attention, conc and offload, not the recipe. So once a sweep finishes, validate_agentic_artifacts reports "agentic point artifacts contain N duplicate row(s)". That breaks the /use <run_id> reuse path validate_reusable_sweep_artifacts.py:59. The artifact names include RECIPE_FINGERPRINT, so uploads don't collide, but the identity check doesn't use it. On the base branch every (tp, conc, offload) point was unique.
Verification: In configs/nvidia-master.yaml, lines 5550 and 5551 differ only in srt-recipe, and so do lines 5552 and 5553. infx/matrix/generate.py:1062-1066 builds the exp_name without the override, so filter_exp_names raises "Experiment name(s) matched multiple rows". agentic_key (infx/results/artifacts.py:169-183) also ignores it, so reuse validation fails. The base had no such duplicates.
| agentic-coding: | ||
| - dram-utilization: 0.683 | ||
| search-space: | ||
| - { tp: 8, spec-decoding: mtp, kv-offloading: none, conc-list: [1], srt-recipe: agentic.yaml } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 5, 10, 15, 20], srt-recipe: agentic.yaml } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: none, conc-list: [15], srt-recipe: agentic.yaml:override_tp4_c15 } | ||
| - dram-utilization: 1.0 | ||
| search-space: | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [30], srt-recipe: agentic.yaml:override_tp4_c30_dram } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [32], srt-recipe: agentic.yaml:override_tp4_c32_dram } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [38], srt-recipe: agentic.yaml:override_tp4_c38_dram } |
There was a problem hiding this comment.
🟡 (optional) Maintainers inherit 11 overrides in the B200 MiniMax-M3 agentic.yaml that no config runs anymore. The master config now names a selector for every point. Overrides left without a selector include override_tp8_c1, override_tp4_c1/c5/c10/c20, and the dram c15/c20/c25/c34/c36/c40 blocks. Fix: delete the overrides no config references, or keep them reachable from a search-space entry. AGENTS.md says to delete recipes and assets no active config uses rather than leave them in place.
Why this was flagged
On the base branch, both search-space groups used bare srt-recipe: agentic.yaml. With no selector, expand_variants in infx/srt_slurm/variants.py:102-104 expands every override_* block, so all of them were reachable. After this diff, nvidia-master.yaml:5527-5553 points each entry at one named override. The only ones used are override_tp4_c15, override_tp4_c30_dram, override_tp4_c32_dram, override_tp4_c38_dram and the six new cap overrides. The only two configs that use srt-recipe-dir minimaxm3/vllm/b200-fp4-mtp are at nvidia-master.yaml:5522 and :5541. That leaves 11 dead blocks in agentic.yaml: override_tp8_c1 (line 56), override_tp4_c1 (69), c5 (82), c10 (95), c20 (121), c15_dram (134), c20_dram (151), c25_dram (168), c34_dram (221), c36_dram (238) and c40_dram (274). They still look like supported benchmark points, but nothing runs or checks them, which conflicts with the AGENTS.md rule to remove unused assets.
Verification: On the base, both search-space groups used bare srt-recipe: agentic.yaml. With no selector, expand_variants (inferencex-e2e/infx/srt_slurm/variants.py) expands every override_* key. After this diff, every entry at nvidia-master.yaml:5527-5553 names exactly one override. That leaves 11 overrides in agentic.yaml that nothing selects.
| # Low-concurrency arms that cap the number of sequences the scheduler keeps active | ||
| # (max-num-active-seqs) below CONC, with a three- or five-token draft. Selected by | ||
| # name from the master config. |
There was a problem hiding this comment.
🟡 nit (optional): The new section comments break the repo's comment policy, which bans header comments above a recipe, override or section. sweep:^# (Low-concurrency arms|TP4 DRAM concurrency 20|CONC 32 with active) Lines 291-293 say what the next overrides are and how they are selected, so they are a section header, not a technical note. Line 377 is the same kind of comment, and its "cut to" wording also describes history. Fix: delete these blocks. If a value needs a reason, keep only a one-line note on that line, for example why max-num-active-seqs sits below CONC.
Why this was flagged
AGENTS.md "Code comments (mandatory)" applies to recipes and configs. It says "Never add broad, overarching comments ... above a recipe, override, or section" and "Do not describe what the file, topology, or config is". It also bans history wording. The diff adds three such comments in agentic.yaml. Lines 291-293 begin "# Low-concurrency arms that cap ... Selected by name from the master config." Lines 333-334 begin "# TP4 DRAM concurrency 20 with 4 and 3 active sequences on the c10 key's image". Line 377 begins "# CONC 32 with active sequences capped below CONC ... the CPU KV pool is cut to the 1,073 GB node allowance." Each one describes the override group below it and does not explain a specific value. The base branch has none of them.
Verification: AGENTS.md lines 60-67 apply to recipes and say: "Never add broad, overarching comments at the top of a file or above a recipe, override, or section." The diff adds such comments in agentic.yaml at lines 291-293 and 333-334, and line 377's "cut to" is history wording. Nothing breaks at runtime; the only effect is a breach of a rule the repository wrote down.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39679190495 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39679190495 |
|
/use 39679190495 |
|
@hshrivastava-droid staged run 39679190495. This run remains available across future |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅Passed and not applicable checks✅ Check 0 (CODEOWNER): PASS — @Ankur-singh is a named owner of ✅ Check 1 (Sweep in PR): PASS — In-PR commit ✅ Check 2 (Evals): PASS — ✅ Check 3 (Upstream recipe): PASS — vllm-project/vllm#63163 is MERGED (2026-10-23, ✅ Check 4 (Reuse command): PASS — ✅ Check 5 (Checklist template): PASS — All current template items are present and checked; the only unchecked item is the conditional "cannot reasonably be satisfied" item, and the additional detail section records the remaining caveats. ✅ Check 6 (Upstream images / ordering): PASS — Both entries are ✅ Check 7 (Deprecated models): PASS — ✅ Check 8 (Architecture hacks): PASS — No ✅ Check 9 (Chat template): PASS — AgentX replays traces against ✅ Check 10 (Engine patches): PASS — No patch, sed, pip or file-overwrite steps in the recipe or config changes. ✅ Check 11 (Golden AL): PASS — The harness injects ➖ Check 12 (Append-only): N/A — The new changelog entry has no ✅ Check 13 (Draft precision): PASS — The draft is ✅ Check 14 (Pareto coverage): PASS — One curve (minimaxm3 / agentic-coding / B200 vLLM / fp4 / run 39679190495 / p90 E2EL vs tput_per_gpu) from 10 raw ✅ Check 15 (PR description): PASS — The body's configs, digest-pinned images, kept/removed points, ✅ Check 16 (NVIDIA overview): PASS — MiniMax-M3 has a vLLM checkmark in the Framework Recipes table of the NVIDIA overview. ➖ Check 17 (Metrics publication): N/A — Direct vLLM serving with no Dynamo TRT-LLM path. Assessed commit: |
…lm-agentic-mtp-retune # Conflicts: # inferencex-e2e/configs/nvidia-master.yaml # inferencex-e2e/perf-changelog.yaml
Summary
Pins the MiniMax-M3 B200 vLLM images by digest, retunes the TP4 concurrency 15 to 38 entries (DRAM KV offload sized to the 1,073 GB node allowance), and adds a second key for arms that cap the number of active sequences.
minimaxm3-fp4-b200-vllm-agentic-mtp,minimaxm3-fp4-b200-vllm-agentic-mtp-c10vllm/vllm-openai:nightly-af7f9488c2210d67e1033ecdc845b087ee7fe92b@sha256:8e2ffbb9f866de678b22e7446903b1ee5a5766c51a948013277477c462a68a77(first key, was tag-only),vllm/vllm-openai:nightly-ac68c3087215e0a4f3cdfa218508c6aada57235d@sha256:98adb2311118263a453ec0acd5d4b72bda3db26cecc0fe8ffc94b9ab03c951f9(new-c10key)minimaxm3-fp4-b200-vllm-agentic-mtp(TP4, EAGLE3 K3)attention-config{"backend":"FLASHINFER","use_trtllm_attention":true,"indexer_kv_dtype":"fp8","minimax_m3_msa_decode_backend":"cutlass"}. The DRAM KV pool is sized to the 1,073 GB node allowance.minimaxm3-fp4-b200-vllm-agentic-mtp-c10(new)Low-concurrency arms that cap the sequences the scheduler keeps active (
max-num-active-seqs) belowCONC, each with its own CUDA graph capture sizes andVLLM_FLASHINFER_ALLREDUCE_BACKEND=mnnvl:Notes
-c10digestsha256:98adb231...resolves to vLLM buildac68c3087215(2026-09-30), and the run logs report that build. The image tag was corrected tonightly-ac68c3087215...to match the digest (it previously named an older commit that predates--max-num-active-seqs); the digest, and so the image that ran, is unchanged.mnnvl) are documented in the vLLM MiniMax-M3 recipe (guide and command builder) by vllm-project/vllm#63163, which is merged.AI model disclosure
claude-sonnet-5-5)PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
配置:
minimaxm3-fp4-b200-vllm-agentic-mtp、minimaxm3-fp4-b200-vllm-agentic-mtp-c10;镜像: 见上方两个vllm/vllm-openai摘要固定镜像。重新调优 B200 vLLM MiniMax-M3 AgentX 在 15 至 38 并发区间的点位(保留 TP4 c15 与 DRAM 卸载 c30/c32/c38,移除其余原有点位),两个镜像均按摘要固定,并新增-c10配置:TP4/TP8 c10(限制活跃序列为 1、K5)以及 TP4 DRAM c20(活跃序列 4 与 3)和 c32(活跃序列 6 与 16)(K3)。-c10镜像摘要对应 vLLM 构建ac68c3087215,标签已更正为与摘要一致(摘要不变)。新增的服务设置已由 vllm-project/vllm#63163 合入上游 vLLM recipe(指南与命令构建器)。AI 模型使用说明:Claude Sonnet 5.5(
claude-sonnet-5-5),负责准备此 PR。改动类型:配置变更。检查清单见上方,除合并前的/use复用评论外均已勾选。